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DoD  Outsourcing  Goals 

1.  Cut  Costs: 

-  Competition  increases  productivity  and  cuts  costs. 

-  Leverage  economies  of  scale  &  scope,  learning  curves, 
and  specialized  human  capital  and  technology  investments. 

2.  Boost  Performance/Effectiveness: 

-  Continuous  improvement  of  product  and  service  quality, 
schedules,  and  responsiveness  to  military  demands. 

3.  Focus  on  Core  Competencies: 

-  Focus  scarce  DoD  resources  and  defense  management 
attention  on  core  competencies. 

-  Provide  oversight  and  monitoring  of  service  and  supply 
contracts,  and  preserve  option  of  future  competitions. 


Examples:  Outsourcing  Travel 

(2005  Vendor  Sales  to  the  Federal  Government) 


Airlines 

2005  ($mil) 

Market  share 

United 

$846 

25% 

Delta 

$718 

21% 

American 

$491 

14.4% 

Cars 

2005  ($mil) 

Market  share 

Hertz 

$76 

20.2% 

Enterprise 

$56 

15% 

Avis 

$54 

14.3% 

Hotels 

2005  ($mil) 

Market  share 

Marriot 

$146 

7.3% 

Holiday  Inn 

$141 

7.0% 

Residence 

Inn 

$125 

6.3% 

Gov’t  Executive 

8/15/06  pp.66-8 

2004 

2005 

Defense 

$8.9bil 

$10.9bil 

Homeland 

Security 

$849mil 

$940mil 

Federal  Outsourcing  Guidance 


Office  of  Management  &  Budget  (OMB  Circular  A-76) 


•  Federal  Acquisition  Streamlining  Act  (PL  103-355) 

•  Federal  Acquisition  Reform  Act  (PL  104-106) 

•  Information  Technology  Management  Reform  Act  (PL 
104-106) 


•  Federal  Activities  Inventory  Reform  Act  (PL  105-270) 


Federal  Acquisition  Regulations  (FAR/DFAR  5000. 1&2) 


_ OMB  Circular  A-76 _ 

•  “Mandates... the  government  obtain  commercially 
available  goods  and  services  from  the  private  sector 
when  it  makes  economic  sense  to  do  so.” 

•  “[R]equires... structured  process  for  [evaluating]  the 
most  efficient  and  cost-effective  method  of 
performance  for  commercial  activities...” 

Four  Steps: 

1 .  Develop  Statement  of  Work  (SOW)  or  Performance  Work  Statement  (PWS)  to 
define  desired  performance/effectiveness  (and  a  Quality  Assurance  Surveillance 
Plan— MOE) 

2.  Construct  Most  Efficient  Organization  (MEO)  for  in-house  competitor 

3.  Issue  Invitation  for  Bid  (IFB)  for  well-defined,  routine  commercial  activities;  or 
Request  for  Proposal  (RFP)  for  less  well-defined,  more  complex  activities 

4.  Source  Selection:  Compare  bids  or  proposals — ’’least  cost”  for  IFB; 


Share  A-76,  Process  Overview, 


“BEST  VALUE ”  for  RFP 


12/22/03 


Federal  Activities  Competed 
Most  Frequently  in  FY  2004-5 


Activity 

FTE 

% 

Maintenance/property 

mgmt 

5,459 

29 

Logistics 

4,435 

23 

Information  technology 

3,262 

17 

HR/personnel  mgmt 

1,378 

7 

Finance  &  accounting 

1,178 

6 

Administrative  support 

1,078 

6 

Other 

2,316 

12 

OMB  (2006)  “Competitive  Sourcing,”  Executive  Office  of  the  President,  p.10 


Outsourcing  Lessons  Learned 

1 .  Use  performance-based  contracting 

-  Do  not  list  tasks  [mix  of  inputs],  but  state  results 
sought  or  problems  to  be  solved  [desired 
attributes/characteristics  of  outputs/outcomes] 

-  Tell  them  WHAT  you  want. . .not  HOW  to  do  it. 

2.  Choose  contractors  according  to  BEST 
VALUE 

-  Source  Selection:  Trade  off  performance  and 
price  instead  of  simply  awarding  to  the  lowest 
bidder. 


J.  Gansler  &  R.  Lipitz  (2003)  “Moving  Toward  Market-Based  Government,”  IBM 
Endowment  for  the  Business  of  Government  (p.  1 5) 


Source  Selection 

•  “Source  selection”  is  the  decision  process  used  in 
competitive,  negotiated,  contracting  to  select  the 
proposal  that  offers  the  Best  Value  to  the 
government.” 

•  In  the  UK  =>  Best  value  =  “Value  for  Money” 

•  In  the  US  =>  “In  different  types  of  acquisitions,  the 
relative  importance  of  cost  or  price  may  vary...” 

FAR  15.101 

•  “This  process  permits  tradeoffs  among  cost/price 
and  non-cost  factors  and  allows  the  Government  to 
accept  other  than  the  lowest  priced  proposal.” 


FAR  15.101-1  (2)c 


Analysis  of  Alternatives  (AoA) 

www.deskbook.osd.mil 


“An  AoA  is  an  analytical 
Comparison  of  the 
operational 
Effectiveness, 
suitability,  and 
Life-Cycle  Cost  of 
Alternatives  that  satisfy 
established 
Capability  needs.” 

Defense  Acquisition  GuideBook  Section  3.3) 


Evaluation  of  Alternatives  (EOA) 


•  Economics  Approach: 

-  Objective:  Select  Alternative  that 

Maximizes  MOE  =  Utility  =  U(non-cost  factors), 
Subject  to  BUDGET  constraint 


•  (MOE)  Build  Effectiveness  model  (non-cost 
factors:  Performance  =  quality,  schedule,  etc.) 

•  Saaty’s  Analytical  Hierarchy  Process  (AHP) 

•  (COST)  Build  Cost  model  (costs/prices) 

•  Estimate  total  system  life  cycle  costs  (total  ownership  costs) 

•  (BUDGET)  Estimate  budget  (constraint) 

•  Construct  Alternatives 

•  In  the  Spirit  of:  Cost  as  an  Independent  Variable  (CAIV)  and 
Target  Costing 


Evaluation  of  Alternatives  (EoA)  Proposal: 

Six  Wavs  to  Structure  an  EoA 

Build  Alternatives:  “Intra-Program  Analysis” 

1.  Fixed  Budget  Approach 

2.  Fixed  Effectiveness  Approach 

3.  Expansion  Path  Approach  (Construct  alternatives  as  Cost- 
Output/Effectiveness  Relations  or  “Response  Functions”) 

Modify  Existing  Alternatives:  “Level  the  Playing  Field” 

4.  Modified  Budget  Approach:  GOTO  1. 

5.  Modified  Effectiveness  Approach:  GOTO  2. 

Cannot  Modify  Existing  Alternatives:  “Inter-Program  Analysis” 

6.  Opportunity  Cost/Benefit  Approach 


Cost-Effectiveness  EoA 

Build  Alternatives 


1.  Fixed  Budget  Approach 

Maximize  Effectiveness  subject  to  Budget  Constraint 
(construct  alternatives  for  given  budget) 


Outsourcing  Opportunity: 

Can  we  get  more  bang  for  the 
same  bucks? 


Benefit/Cost 


Cost-Effectiveness  EoA 

Build  Alternatives 


2.  Fixed  Effectiveness  Approach 

Dual:  Minimize  Costs  subject  to  Effectiveness  Constraint 

(construct  alternatives  forgiven  MOE) 


Benefit/Cost 


Outsourcing  Opportunity: 

Can  we  spend  less  bucks  for 

the  same  bang? 

QMB  Circular  A-76:  Statement  of 
Work(SOW);  Invitation  for  Bid(IFB) 

If  A1  is  the  In-house  MEO,  then 
“Outsourcing’s  Out  &  Insourcing’s  In” 

(DoD  FY  2005  :  71%  A-76  won  by  in-house  MEO) 


Cost($) 


AoA 


“Typically,  the  last  analytical  section  of  the  AoA  plan 
deals  with  the  planned  approach  for  the  cost- 
effectiveness  comparisons  of  the  study  alternatives.” 

[THIS  IS  FIRST  STEP  OF  Economic  Approach  to  EoA] 

•  “Cost  effectiveness  comparisons  in  theory  would  be 
simplified  if... all  the  alternatives  have  equal  effectiveness  (the 
best  alternative  is  the  one  with  the  lowest  cost)  or  equal  cost 
(the  best  alternative  is  the  one  with  the  greatest  effectiveness).” 

•  “In  actual  practice,  the  ideal  of  equal  effectiveness  or  equal 
cost  alternatives  is  difficult... to  achieve...” 

•  “A  common... comparison  is  a  scatter  plot  of  effectiveness 
versus  cost.” 


Defense  Acquisition  Guidebook  Section  3.3  www.deskbook.osd.mil 


Decision  Sciences  Approach 

Identify  “Efficient  Set” 

Weed  out  dominated  alternatives 


MOE(Utils) 


► 


Cost 


Decision  Sciences  Approach 

Those  remaining  form  an  efficient  set 
Which  Alternative  is  the  Best? 


MOE(Utils) 


G n 
G n 
<D 

<D 

> 

• 

o 

w 


A1 

o 


o 


o 


A2 


o 


Efficient  Set 


o 


Cost 


Source  Selection 


“One  straightforward  method  for  combining  cost  and 
effectiveness  involves  constructing  a  ratio.”  (p.6-3) 

“The  methods  we  choose  for  combining  effectiveness 
[and]  cost... depend  upon  the  nature  of  the  problem. 

We  can  fix  either  cost  or  effectiveness.” 

“If  neither  can  be  fixed... we  can  establish  a 
cost/effectiveness  ratio. . .”  (p.6-10) 

C.  Murray,  Editor  (2002)  Executive  Decision  Making,  National  Security  Decision  Making  Dept., 

U.S.  Naval  War  College  (  )  6th  Edition 

Danger  in  applying 
Bang/Buck  or  Buck/Bang 
Ratios  to  rank  alternatives 


Is  A1  really  superior  to  A2  ? 

LESSON:  DANGER  in  using  Benefit/Cost  (Bang/Buck)  or 
Cost/Benefit  (Buck/Bang)  ratios  without  anchoring  Budget  or  MOE 


MOE(Utils) 


G n 


o 


Benefit/Cost 

/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 

/ 

'  / 
/  / 


RANKING: 

(A1)  >  Benefit/Cost  (A2) 

/ 

/ 


✓ 

✓ 

/ 

/ 

/ 


Efficient  Set 


Marginal  Benefit/Marqinal  Cost 

“The  perceived  benefits  of  the 
higher  priced  proposal  shall 
merit  the  additional  cost...” 

FAR  15.101-1  (2)c 


Cost 


Source  Selection 

“The  solicitation  shall  state  whether  all  evaluation 
factors  other  than  cost/price,  when  combined 
[MOE],  are  significantly  more  important  than, 
approximately  equal  to,  or  significantly  less 
important  than  cost/price.” 


FAR  15.101-1(2) 

“[AJgencies  must:  a)  identify  the  specific  weight 
given  to  each  evaluation  factor..., and  b)  make  the 
specific  weight  for  cost  or  price  at  least  equal 

to  all  other  evaluation  factors  combined. . .” 


OMB  (2006)  “Competitive  Sourcing,”  Executive  Office  of  the  President,  p.21 


So  Which  Alternative  is  “Best”? 

Decision  Sciences  Approach 

Max  V  =  V(MOE,Cost)  =  w1*MOE  -  w2*Cost 

Ask  Decision  Maker  What  is  More  Important:  MOE  or  Cost? 

(dMOE/dCost  =  w2/w1 ) 


Cost 


Which  Alternative  is  “Best”? 

Decision  Sciences  Approach 

Max  V  =  V(MOE,Cost)  =  w1*MOE  -  w2*Cost 


MOE(Utils) 


G n 
G n 
<D 

<D 

> 

• 

o 

w 


(dMOE/dCost  =  w2/w1) 


A2 


If  Performance  has  a 
sufficiently  greater  weight 

(w1»w2),  then  high  cost, 
high  effectiveness 
alternative  A2  wins 


Cost 


“Most  Common  Critical  Mistake” 


“One  mistake  is  very  commonly  made  in  constructing 
value  models... illustrated  in  the  context  of... air 
pollution.  ..[i.e.  reducing  pollutant  concentrations] 

I  personally  do  not  want  some  administrator  to 
give  two  minutes  of  thought  to  the  matter  and 
state  that  [reducing]  pollutant  concentrations  [is] 
three  times  as  important  as  cost  ” 


DECISION  SCIENCES  EXPERT 

R.  Keeney  (1994)  “Using  Values  in  Operations  Research,”  Ops.  Research  42/5  p.797 


Question:  How  does  Decision  Maker  (DM)  decide  relative 
weights  to  assign  to  MOE  &  COSTS?  What  does  this  mean? 

•  Economist’s  Hypothesis: 

-  If  DM  cares  about  COSTS  (i.e.  places  any  weight  on 
cost)  it  is  because  there  is  a  budget  constraint  or 
opportunity  cost  of  obtaining  funds  to  pay  for  the  extra 
MOE  in  this  program. 

-  Otherwise  “Go  for  the  Gusto  (greatest  MOE)!” 

-  Decision  Sciences  (AoA)  approach  addresses  this  indirectly: 

Typical  Objectives  Hierarchy  includes  both  Max  MOE  &  Min  Costs  => 

Max  V  =  V(MOE,Cost)  =  w1*MOE  -  w2*Cost 

-  One  ubiquitous  source  of  confusion  is  the  attempt  to 
maximize  gain  while  minimizing  cost... if  a  person 
approaches  a  problem 
[decision]  criterion,  he 


with  the  intention  of  using  such  a 
is  confused  to  begin  with...”  (p.167) 

Hitch  C.  &  R.  McKean  1967  “The  Economics  of  Defense  in  the 
Nuclear  Age”  Harvard  University  Press,  Cambridge,  Mass. 


Economics  EOA  Approach 

Evaluation  of  Alternatives  (EOA): 

-  “[A]  criterion  in  which  the  budget  or  level  of  effectiveness  is 
specified  has  the  virtue  of  being  aboveboard.”  (p.  167) 
[EoA  approaches  1 .  and  2.] 


-  “The  test  of  maximum  effectiveness  for  a  given  budget 
seems  much  less  likely  to  mislead  the  unwary... ”(p. 167) 

-  “As  a  starter,... several  budget  sizes  can  be  assumed. 

-  If  the  same  [alternative]  is  preferred  for  all... budgets,  that  system  is 
dominant. 

-  If  the  same  [alternative]  is  not  dominant,  the  use  of  several... budgets  is 
nevertheless  an  essential  step,  because  it  provides  vital  information  to 
the  decision  maker.”  (p.176) 

Hitch  C.  &  R.  McKean  1967  “The  Economics  of  Defense  in  the 
Nuclear  Age”  Harvard  University  Press,  Cambridge,  Mass. 


EOA  Proposal:  Two-Stage  Optimization 

“Tell  them  WHAT  you  want  and  roughly  what  you  can  afford,  then  let  them  figure  out  HOW  to  do  it.” 

i)  First  Stage:  (CAIV) 

-  DoD  provides  notional  budget  guidance  (B)  to  alternative  vendors  for  the 
program.  DoD  searches  for  the  optimum  product  (Procurement)  and/or 
service  (R&D;  O&M)  package  it  can  obtain  at  that  price,  B.  DoD  also  reveals 
optimistic  and  pessimistic  budget  guidance. 

-  DoD  defines  the  set  of  characteristics/attributes  it  values  and  this  is 
known  to  vendors,  but  DoD’s  precise  Utility  Function  over  those 
characteristics  is  unknown  to  vendors. 

ii)  Second  Stage:  (Target  Costing) 

-  Vendors  have  different  costs  and  production  functions  for  generating 
products  or  services  (defined  as  bundles  of  characteristics). 

-  Each  vendor  maximizes  its  output  offer  (an  optimal  mix  of  the  desired 
characteristics)  subject  to  their  particular  budget  constraint  (which 
includes  DoD’s  budget  guidance  and  the  vendor’s  individual  costs  to  produce 
a  unit  of  each  characteristic). 

-  This  is  the  product  and/or  service  package  (output)  a  particular  vendor 
is  able  to  propose  for  each  possible  budget  (B),  given  their  production 
function  (technical  production  possibilities)  and  their  costs  of  generating  those 
characteristics. 

-  With  the  latest  budget  forecast,  DoD  selects  among  the  optimized 

characteristic  bundles  proposed  by  each  vendor,  the  bundle/alternative 
(total  product/service  package)  that  maximizes  DoD’s  Utility  Function. 


DECISION  SCIENCES  APPROACH:  Exogenous  Alternatives 
Example  of  “SUPERIOR  (Dominant)  SOLUTION” 


Source  Selection  Decision:  Eliminate  A2 ! 


▲ 

MOE(Utils) 


C/3 

xn 


o 


► 

Budget  ($) 


ECONOMIC  APPROACH:  Evaluation  of  Alternatives  (“Engel  Curves”) 
3.  Expansion  Path  (Response  Function)  Approach 
Do  not  eliminate  A2  prematurely:  Explore  impact  of  budget 

cuts  (Identify  vendor  responses) 


▲ 

MOE(Utils) 


G n 


o 


A2 


Bl 


► 

Budget  ($) 


ECONOMIC  APPROACH:  Endogenous  Alternatives  (“Engel  Curves”) 
3.  Expansion  Path  (Response  Function)  Approach 
(Alternatives  are  Cost-Effectiveness  Relations,  not  Points) 
Explore  impact  of  budget  cuts  (Identify  vendor  responses) 


Source  Selection  Decision:  A2  for  pessimistic  budget;  A1  for  optimistic  budget 


What  if  we  cannot  build  Alternatives? 

(Alternatives  have  already  been  identified) 

•  “In  many  cases,  there  will  be  a  minimum  set  of 
alternatives  required  by  the  initial  analysis 
guidance.” 


•  In  most  AoAs,... comparisons  involve  alternatives 
that  have  both  different  effectiveness  and  cost, 
which  leads  to  the  question  of  how  to  judge  when 
additional  effectiveness  is  worth  additional  cost. 


(Defense  Acquisition  GuideBook  Section  3.3 
www.deskbook.osd.mil) 


EOA:  “LEVEL  THE  PLAYING  FIELD” 

4.  Modified  Budget  Approach  (GOTO  1  &  3) 
Modify  alternatives  to  equalize  budget 


(Identify  vendor  MOE  responses  to  budget  increase) 


EOA:  “LEVEL  THE  PLAYING  FIELD” 


5.  Modified  Effectiveness  Approach  (GOTO  2  &  3) 
Modify  alternatives  to  equalize  MOE 
(Identify  vendor  COST  responses  to  higher  MOE  requirement) 


6.  Opportunity  Cost  Approach 

(INTER-PROGRAM  Analysis) 

What  if 

-We  Cannot  Modify  alternatives  to  obtain 
response  functions? 

and 

-We  don’t  know  or  cannot  assume  a  given 
Budget  or  desired  MOE. 

Then  some  alternatives  (bundles)  cost  more  but 
offer  more  effectiveness,  while  others  cost  less 
and  offer  less  effectiveness  (“efficient  set”). 


MOE 


Example:  Evaluate  Alternative  Radar 

Maintenance  Packages 

0.70 
0.60 
0.50 
0.40 
0.30 
0.20 
0.10 
0.00 


Rad 

Radar  2 

o 

Radar  1 

o 

Radar  4 
ar  3  q 

$0  $20  $40  $60  $80 

Discounted  LCC 


MOE  =  f(Availability;etc.) 


Which  is  “Best  Value?”Radar  1  or  2? 

(MOE  =  f(Availability;etc.)) 


0.70 


0.60 


0.50 


0.40 


0.30 


0.20 


0.10 


0.00 


Rad 

Radar  2 

Radar  1  T 

Radar  4 
ar  3  q 

o 

Attributes  Radar  1  Radar  2 

Change 

Availability 

0.8 

0.9 

'  ai 

COST  |  $14.00  |  $35.00 

$21.00 

$0  $20  $40  $60 

Discounted  LCC 


$80 


Is  it  worth  $21  mil  over 
the  life  of  the  system  to 
obtain  the  extra 
Availability  MOE? 


6.  Opportunity  Cost  Approach 

(INTER-PROGRAM  Marginal  Analysis) 

A)  Question:  Where  is  the  extra  money  coming  from  if  I 
buy  the  high  cost  alternative? 

B)  Question:  Where  is  the  extra  money  going  if  I  buy  the 
low  cost  alternative? 


A 


MOE(Utils) 


Program  A 


MOE(Utils) 


Program  B 


A2 

o 


B2 

o 


A1 

o 


B1 

o 


Cost($) 


Cost($) 


AoA 

[Marine  Corps  Systems  Command  PA&E  Methodology] 

Statement  of  requirements 

-  Mission  Needs  Statement  (“Customer”),  Subject  Matter  Expert  input,  etc. 
Development  of  alternatives 

-  Complete  and  exhaustive;  Think  broadly 

Multi-step  approach;  eliminate  no  alternative  before  its  time 

-  “Do  nothing”  is  an  alternative! 

Evaluation  of  effectiveness  of  alternatives  (Modeling  &  Simulation) 

-  Performance  (MOP):  inherent  characteristic  of  alternative  ^ 

-  Effectiveness  (MOE):  contribution  of  alternative  to  overall  mission^!/ $ 
Estimation  of  “rough  order  of  magnitude”  life-cycle  costs 


Integration  of  results 

Equal  effectiveness 

•  Set  a  threshold  for  a  given  level  of  effectiveness 

•  Buy  enough  systems  to  achieve;  compare  costs 
-  Equal  cost 

•  Set  a  fixed  expenditure  rate 

•  Compare  effectiveness  with  equal-cost  alternatives 


Conclusions  and  recommendations 
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Evaluation  of  Alternatives  (EEoA)  Proposal: 

Six  Wavs  to  Structure  an  EEoA 

Build  Alternatives:  “Intra-Program  Analysis” 

1.  Fixed  Budget  Approach 

2.  Fixed  Effectiveness  Approach 

3.  Expansion  Path  Approach  (Construct  alternatives  as  Cost- 
Output/Effectiveness  Relations  or  “Response  Functions”) 

Modify  Existing  Alternatives:  “Level  the  Playing  Field” 

4.  Modified  Budget  Approach:  GOTO  1. 

5.  Modified  Effectiveness  Approach:  GOTO  2. 

Cannot  Modify  Existing  Alternatives:  “Inter-Program  Analysis” 

6.  Opportunity  Cost/Benefit  Approach 


Enjoy  your  stay  in  Monterey! 
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DoD  Outsourcing  Examples 

Materiel  management 

-  Distribution 

-  Inventory  control 

-  Disposal 

Base  commercial  activities 

-  Facilities  maintenance 

-  Food  services 

-  Vehicle  maintenance 
Depot  maintenance 
Finance  &  accounting 

-  Purchasing  &  Travel  (credit)  cards 

-  Debt  &  claims  management 

-  Administrative  support 
Training 

-  Simulators 

-  Distance  Learning 
Information  Technology 

-  Computer  equipment,  maintenance  &  repair 

-  Communication  equipment,  maintenance  &  repair 

-  Software  development 

-  Internet  services 


Summary:  Six  Approaches  to  AoA 

Can  Construct  alternatives: 

1 .  Fixed  Budget  Approach  (Construct  alternatives  with  equal 
budget): 

-  Objective:  Maximize  Effectiveness 

2.  Fixed  Effectiveness  Approach  (Construct  alternatives  with  equal 
effectiveness): 

-  Objective:  Minimize  Costs 

3.  Expansion  Path  Approach  (Construct  alternatives  as  Cost- 
Output/Effectiveness  Relations):  Sensitivity  Analysis 

Can  Modify  pre-determined  alternatives:  “Level  the 
Playing  Field” 

4.  Modified  Budget  Approach  (Identify  high/low  cost  alternative  as 
“revealed”  budget  constraint  and  adjust  others  accordingly) 

-  Objective:  Maximize  Effectiveness  (GOTO  1  &  3) 

5.  Modified  Effectiveness  Approach  (Identify  effectiveness  of  an 
alternative  as  “revealed”  objective  and  adjust  others  accordingly) 

-  Objective:  Minimize  Costs  (GOTO  2  &  3) 

Cannot  Construct  or  Modify  alternatives: 

6.  Opportunity  Cost/Benefit  Approach:  (Inter-program  analysis) 


To  generate  MOE: 

Identify:  decision  scenario,  relevant  players 
“decision  makers”  (DM),  and  time  frame 

•  Identify  relevant  players:  DM 

-  Users 

-  Evaluators  MOE 

-  “Political”  Stakeholders 

—  Payers  $$  (Cost  as  an  Independent  Variable — CAIV  ) 

•  “Top-down”  approach  to  assist  DM  to 
describe  components  of  MOE  (utility)  function 

-  Saaty’s  (1977)  Analytic  Hierarchy  Process  (AHP) 

-  Proceed  from  general  criteria  to  measurable  attributes 


Example:  MOE 


